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Promise  and  Pitfalls  of  Prediction  Markets 


•  Potentially  VERY  powerful  tool 
BUT  ... 


•  The  “devil  is  in  the  details” 


-  Small  changes  in  market  design  can  have  big  impacts 

-  #1  rule  of  carpentry:  “Measure  twice,  cut  once” 


•  A  little  econ  knowledge  can  be  a  dangerous  thing 


-  A  basic  understanding  of  the  principles  is  not  sufficient 
to  design  and  implement  an  effective  market 

-  Within  DoD,  several  examples  where  a  good  economic 
idea  has  been  implemented  in  a  sub-optimal  fashion 


-  Pattern:  From  “too  slow”  to  “too  fast”  with  good  ideas 
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Popular  (but  Applicable?)  Prediction  Markets 


Market 


Focus 


Iowa  Electronic  Markets 
<www.biz.iowa.edu/iem> 

Run  by  University  of  Iowa _ 

Centrebet 

<www.centrebet.com> 

For  profit  company _ 

intrade 

<www.intrade.com/> 

For  profit  company _ 

Economic  Derivatives 
<www.economicderivatives.com> 

Run  by  Goldman  Sachs  and  Deutsche  Bank 
Newsfutures 

<www.newsfutures.com> 

For  profit  company _ 

Foresight  Exchange 
<www.ideosphere.com> 

Non-profit  research  group _ 

Hollywood  Stock  Exchange 
<www.hsx.com> 

Owned  by  Cantor  Fitzgerald 


Small-scale  election  markets. 


Northern  Territory  bookmaker,  offering  odds  on 
election  outcomes,  current  events,  sports,  and 
entertainment. _ 

Trades  in  political  futures,  financial  contracts, 
current  events,  entertainment,  etc. 


Large-scale  financial  market  trading  in  the  likely 
outcome  of  future  economic  data  releases. 


Political,  finance,  current  events  and  sports  markets. 
Also  technology  and  pharmaceutical  futures  for 
specific  clients. _ 

Political,  finance,  current  events,  science  and 
technology  events  suggested  by  clients. 


Success  of  movies,  movie  stars,  and  awards.  Data 
used  for  market  research. 
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Applications  Relevant  for  Acquisition 


Application  Firm 


Description 


Sales 

Forecasting 

Hewlett- 

Packard 

Hewlett-Packard  used  an  internal  market  system  to 
forecast  printer  sales.  HP’s  official  forecast  erred  by  13%, 
while  the  market  erred  by  6%;  the  market  performance 
exceeded  the  accuracy  of  official  forecasts  75%  of  the  time. 

Product 

Development 

Eli  Lilly 

Eli  Lilly  applied  internal  markets  to  predict  which  of  six 
potential  new  drugs  would  have  the  greatest  success  in 
passing  product  development  hurdles. 

General 

Forecasting 

Google 

Google  uses  internal  markets  to  forecast  new  product 
launch  dates  and  new  office  openings,  predicting  the 
likelihood  that  an  event  will  occur  and  on  a  specific  date. 

Product 

Development 

Siemans 

An  internal  market  predicts  whether  the  firm  will  deliver 
software  projects  on  time 

Product 

Development 

Microsoft 

Uses  internal  markets  to  predict  whether  projects  will  meet 
milestones  articulated  in  their  project  plans 
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Sources:  Chen  and  Plott  (2002),  Corporate  Executive  Board  2006),  Kiviat(  2004),  Malone  (2004),  Ortner  (1998) 


Asset  Value  &  Market  Prices  as  Predictors 


•  Two  dimensions  of  value  for  any  asset/contract : 

-  Arbitrage  value:  Potential  financial  benefit  from 
reselling  asset/contract  at  a  higher  price  at  a  later  date 

-  Intrinsic  value:  Expected  financial  benefit  of  holding 
the  asset/contract  indefinitely  (or  until  market  closes) 

•  Many  people  think  about  the  value  of  stocks  in  the 
stock  market  only  in  terms  of  the  arbitrage  value 


•  Prediction  markets  provide  reasonable  predictions 
only  when  prices  reflect  estimate  of  intrinsic  value 


•  Thus,  assets  and  markets  must  be  designed  to 
direct  focus  toward  intrinsic  value 
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Measuring  Prediction  Market  Performance 


•  Prediction  markets  make  predictions  by 
aggregating  knowledge  among  traders 

•  Thus,  prediction  market  performance  can  only  be 
measured  by  degree  of  information  aggregation 


•  Prediction  market  performance  can  not  best  be 
measured  by  prediction  accuracy 

-  Predicting  a  coin  flip 


-  Better  weatherman:  Las  Vegas  or  Seattle? 


•  At  best,  prediction  market  performance  can  be 
measured  by  relative  accuract 


-  But  alternative  predictions  often  don’t  exist 
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Measuring  Prediction  Market  Performance 


•  “There’s  a  common  ‘weatherman’  misunderstanding  about 
prediction  markets,  especially  in  the  press.  Perhaps 
counterintuitively,  a  weatherman  is  not  wrong  if  the  sun 
comes  out  after  a  90  percent  forecast  for  rain  because 
there  was  still  a  10  percent  chance  of  sunshine.  Instead, 
the  weatherman  is  a  good  predictor  if  it  rains  90  percent  of 
the  time  when  he  gives  a  90  percent  chance  of  rain — any 
more  or  less  would  be  poor  predictions.  Prediction 
markets  work  the  same  way.” 

•  -Todd  Proebsting,  Microsoft 
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Prediction  Markets  as  a  Decision  Tool 


PROBLEM: 

•  Consider  asset  measuring  project  performance 

•  Market  price  drops  (negative  reflection  on  project) 
=>  DoD  or  agency  takes  corrective  action 

=>  Market  price  goes  up  ...  or  does  it? 

•  Market  price  should  already  incorporate 
expectations  about  any  corrective  action 

=>  Market  price  may  not  reflect  what  you  think 


SOLUTIONS: 

•  Measure  variables  unaffecting  by  DoD  decisions 

•  Conditional  assets/contracts:  If  .  then 
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Using  Conditional  Assets/Contracts 


DESCRIPTION: 

•  Conditional  assets  ask: 

-  “If _ ,  then  what  is  the  probability  that _ ?” 

-  “If _ ,  then  what  will  be  the  measure  of _ ?” 

PROBLEM: 

•  How  do  assets  “payoff”  if  condition  doesn’t  occur? 

•  If  zero  value,  distorts  price  as  a  predictor 
SOLUTION: 


•  Closing  price  termination  rule 

•  If  an  asset  is  “delisted,”  each  share  of  asset  pays 
amount  equal  to  average  price  over  final  X  days 

-  Delisting  fear  should  not  distort  prediction  value  of  price 
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Incentive  Issues  with  “Play  Money” 


•  Private  corporations  use  real  money 

•  Resistance  to  using  real  money  in  government 

•  With  play  money,  traders  go  for  the  home  runs: 

-  Buy  low-probability  events 

-  Avoid  high-probability  events 

-  Leads  to  inaccurate  probabilities  of  events 

•  Research  suggests  real  money  gives  more 
accurate  predictions 
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Incentive  Issues  with  “Play  Money” 


•  Competition  and  bragging  rights  as  an  incentive? 

-  Requires  degree  of  non-anonymity  (not  good) 

-  How  do  traders  know  if  they  are  “winning?” 

-  Intrinsic  values  not  realized  until  after  close  of  market 


•  Portfolio  value  as  measure  of  performance? 

-  Does  not  recognize  good  predictors  of  outcomes 

-  Rewards  traders  for  being  a  good  predictor  of  other 
trader’s  predictions  (of  other  trader’s  predictions  ...) 


•  Beware  of  research  suggesting  play  money  works 


-  Settings  often  involve  intrinsic  motivation 
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Market  Maker  Concerns 


•  Thin  market  =>  asynchronous  trades  =>  market  maker 

•  BUT  how  is  price  adjusted  in  response  to  trades? 

-  Critical  for  price  to  be  a  reasonable  predictor 

-  Without  knowledge  of  underlying  supply  and  demand, 
price  adjustment  is  educated  “guesswork” 

•  Current  price  may  not  reflect  aggregate  belief 

-  Contributes  to  price  volatility 


•  Alternative  approach:  Proxy  bid/ask  thresholds 


-  If  price  rises  above  $ _ ,  then  sell _ units 

-  If  price  falls  below  $ _ ,  then  buy _ units 

-  PLUS:  Gives  off-equilibrium  supply  and  demand  info 
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Asset/Contract  Types 


•  “Winner-take-all”  assets/contracts 


-  Pays  fixed  amount  if  outcome  occurs,  zero  otherwise 

-  Price  reflects  market  expectation  of  probability  of  event 

•  “Index”  assets/contracts 


-  Pays  variable  amount  tied  to  a  specific  future  measure 

-  Price  reflects  expectation  of  mean  value  of  measure 


•  “Spread”  assets/contracts 


-  Pays  fixed  amount  if  &  only  if  measure  is  above/below 
threshold  (adjusted  to  balance  two  sides  of  market) 

-  Price  reflects  expectation  of  median  value  of  measure 
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Asset/Contract  Types 

•  Winner-take-all  markets  are  easiest  to  implement 

-  Allow  use  of  market-maker  algorithm  for  thin  market 

-  Widely  available  mechanisms  for  doing  so 

BUT  ... 

•  Index  markets  may  reveal  more  valuable  info 

-  Specific  date  on  which  an  event  is  expected  to  occur 

-  Specific  degree  of  cost-growth  that  is  expected  to  occur 
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Asset/Contract  Design  Issues 

•  To  be  valuable,  a  prediction  market  asset  must 
measure  something  that  is: 

-Valuable 

-  Quantifiable 

-  Clearly  defined 

-  Ex-post  measurable 

-  Exogenously  determined 
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Thesis  Student  Pilot  Study 


•  Participants: 


-  N1  personnel 

-  Recruiting  command  personnel 

-  Researchers  at  NPS,  CNA,  other  research  org’s 

•  Questions: 


-  Manpower  outcomes 

-  Generally  economic  outcomes  (unemployment,  stock 
market) 

-  Fun  questions  (NFL  opener,  Emmy’s,  MLB  play-offs) 
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Thesis  Student  Pilot  Study 

•  Participation  was  very  low  and  tailed  off,  despite 
encouragement  from  N1 

-  1 1  of  35  non-NPS  potentials  made  trades 

•  Adding  “fun”  questions  didn’t  seem  to  increase 
participation  in  manpower  outcomes 

•  Top  reasons  for  lack  of  participation: 

-  Lack  of  time 

-  No  incentive 
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Pilot  Experiment 
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Attribute  3 


Market  Price 


Pilot  Experiment  Results 

Price  Patterns  &  Performance  Indicators 

Traders  25  24  23 
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1.20  i 1 1 i 1 1 1 1 1 1 1 1 ! 1 1 1 1 ! » 1 1 1 1 1 


Market  Price 


Pilot  Experiment  Results 

Price  Patterns  &  Performance  Indicators 


Traders  25  24  23  22  25  15  23  21  24  22 

Period  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 


Pilot  Experiment  Results 

Price  Patterns  &  Performance  Indicators 


Traders  25  24  23  22  25  15  23  21  24  22  18  13  17  12  14  15 

Period  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 


Pilot  Experiment  Results 

Price  Patterns  &  Performance  Indicators 


Traders  25  24  23  22  25  15  23  21  24  22  18  13  17  12  14  15  11  10  11  12  10 

Period  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 


Pilot  Experiment  Results 
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Information  Aggregation  (Kaiovcova  2007) 


•  Six  equally  likely  possible  states 

—  A,  B,  F 
-  True  state:  A 


Five  individuals  have  private  signal  of  true  state 

-  Represented  by  balls  in  an  urn 

•  A  represented  by  five  balls;  B  -  F  by  two  each 

Draws-  aab  aee  abf  acd  cdf 

i  i  I  I  I  * 

A  E  ?  ?  ? 

Aggregate  Information:  AAAAA,  BB,  CC,  DD, 


EE,  FF 
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HP  found  results  promising  for  using  prediction 
markets  to  predict  future  sales  com 

HP  and  Siemens  found  motivating  employees  to 
trade  a  major  challenge 

Microsoft  limits  participation  to  informed  traders; 
uninformed  traders  are  less  likely  to  participate 
(risks  omitting  informed  trader) 
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Iowa  Electronic  Markets 

Presidential  Elections  1988  -  2000 


Days  l  mil  l  leciion 

Source:  Wolfers  and  Zitzewitz  2004 
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Anatomy  of  prediction  markets: 

•  ConMi9§i^fflnKW¥iiJra#ing  mechanisms 

-  Person-to-person  selling 

-  Needs  to  be  a  seller  for  a  person  who  wants  to  buy 

-  Examples:  Intrade,  Iowa  Electronic  Markets 

•  Market  Scoring  Rules  Mechanism: 

-  An  algorithm  determines  prices  based  on  demand  for 
various  outcomes 

-  Needed  when  participation  is  low 

-  Example:  Inkling  Markets 
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Most  common  types  of  securities 


•  Winner  take  all  for  an  event  occurring: 

-  Price  represents  probability  of  event  occurring 

-  Sarah  Palin’s  price  for  being  2012  Republican 
nominee  is  between  24.4  (bid)  and  25.0  (ask) 

•  Index 


-  For  continuous  variables 

•  What  will  Dow  Jones  be? 

-  Intervals  often  used  instead: 

•  N1  Marketplace:  What  was  the  unemployment  rate  going 
to  be  for  Sept.  2009? 

-  Separate  securities  for: 

»  <9%,  9-9.3%,  9.4 -9.7%,  ... 
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Limitations 


•  Participation  will  often  be  limited 

-  Small  #  experts  for  most  outcomes 

-  Many  potential  traders  don’t  understand  the  market 

-  Traders  will  fear  some  people  have  huge  advantage 
from  insider  info 


•  E.g.,  Those  who  have  authority  to  set  SRBs 

-  But,  uninformed  traders  are  needed  for  knowledgeable 
traders  to  make  money  and  have  incentive  to  play 


-  Unlike  with  stock  market,  there  can  be  long  periods  of 
time  with  no  activity 
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Limitations 

•  Thin  markets  (from  low  participation)  cause: 

-  Less  accurate  predictions 

-  More  volatility 

-  Need  for  less  ideal  Market  Scoring  Mechanism 

•  Traders  often  don’t  understand  short-selling 

-  So  difficult  for  people  to  bet  against  something 
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Limitations 


•  Manpower  outcomes  can  be  self-defeating 

-  Suppose  there’s  a  market  for  whether  we  meet 
endstrength  goal 

-  Trader  1  makes  a  correct  assessment  that  Navy 
won’t  meet  goal  and  sells  shares 

-  Price  goes  down 

-  Navy  sees  low  probability  of  meeting  goal 


-  Navy  ups  reenlistment  bonuses 

-  Now  Trader  1  will  lose  money 

-  (Are  there  similar  potential  problems  for 
Acquisitions?) 
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Lessons  learned:  need  to  phrase 
questions  very  clearly 


•  Inkling  question:  Did  Juliet  die  on  Lost?  Question  will 
close  January  28,  2010  @  09:34pm  PST 


-  Nope 

-  Yes.. ..well  as  far  as  we  know 

-  Sort  of  but  reincarnated 

-  Who  knows  -  Alternative  reality 


$26.51/ $100.00  (closed) 

$17.36/ $0.00  (closed) 
$21.19/ $0.00  (closed) 
$34.94/  $0.00  (closed) 


•  Our  question,  posed  August  7,  2009: 

-  Will  the  Dow  Jones  Industrial  Average  (INDU)  close  above 
9,400  by  COB  on  Friday,  Aug.  14,  2009? 

-  Dow  closed  at  around  9395  on  August  12 
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